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APPARATUS AND METHOD FOR MULTIPLE DESCRIPTION ENCODING 



CROSS-REFERENCE TO RELATED APPLICATION 

[0001] This application claims the benefit of U.S. Provisional Patent Application 

Serial No. 60/426,887, filed November 15, 2002, which is incorporated herein by 
reference in their entirety. 



BACKGROUND 



I. Field of Invention 

[0002] The invention generally relates to multi media and more particularly to 

discrete cosine transform based compression system for generating multiple 
descriptions of data. 



II. Description of the Related Art 

[0003] Typically, digital information is compressed using a pre-selected format or 

process by an encoder. However, conventional digital consumer formats such as High 
Definition Television (HDTV), Digital Versatile Disc or Video Disc (DVD), 
Advanced Television Systems Committee (ATSC), Digital Video Broadcast (DVB), 
Digital Satellite System (DSS) operate at various specific resolutions, frame rates 
and/or bit rates. Accordingly, in order to cater to the various formats, a compression 
technique that can generate multiple descriptions of video is required. 

[0004] Current video compression standards that provide multiple descriptions do so 

in an innate form or are targeted toward a discrete application. For example, Joint 
Photographic Experts Group (JPEG) 2000 can generate multiple descriptions of video 
by reducing the video. However, being intraframe and wavelet based, JPEG 2000 
inherently provides lower resolution images. Moreover, it is restricted to being 
dyadic, i.e. reduction factors are multiples of two. Motion Picture Experts Group 
(MPEG) 4 also supports multiple description discrete cosine transform (DCT) that 
targets limited or fixed bandwidth applications such as Internet video. In this 
technique, a rudimentary form of the video is transmitted. Consecutive transmissions 
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are made to enhance the details (bits) in the video. A major disadvantage in this 
approach is motion compensation. 
[0005] Therefore there is a need for a more versatile, simple and/or efficient system 

that can generate multiple descriptions of video or video sequences. 

SUMMARY 

[0006] Embodiments disclosed herein address the above stated needs by providing a 

system for generating multiple descriptions of compressed data. In one embodiment, 
a method for generating multiple descriptions of compressed data comprises 
generating transform coefficients from input data; quantizing the transform 
coefficients; generating an energy distribution of the quantized transform coefficients; 
grouping the transform coefficients into layers based on the energy distribution; and 
entropy coding a first number of layers to generate a first description of compressed 
data. 

[0007] In another embodiment, an apparatus for generating multiple descriptions of 

compressed data comprises means for generating transform coefficients from input 
data; means for quantizing the transform coefficients; means for generating an energy 
distribution of the quantized transform coefficients; means for grouping the transform 
coefficients into layers based on the energy distribution; and means for entropy 
coding a first number of layers to generate a first description of compressed data. 

[0008] In still another embodiment, an apparatus for generating multiple descriptions 

of compressed data comprises a transform module configured to generate transform 
coefficients from input data; a quantization module coupled to the transform module 
and configured to quantize the transform coefficients; a layering module coupled to 
the quantization module, the layering module configured to generate an energy 
distribution of the quantized transform coefficients and to group the transform 
coefficients based on the energy distribution; and an entropy coder coupled to the 
layering module and configured to entropy code a number of layers to generate a 
description of compressed data. 

[0009] In the above embodiments, the transform coefficients may be grouped in an 

order of significance. The transform coefficients may be grouped by splitting the 
transform coefficients into multi-bit units; and grouping the multi-bit units into layers. 
The transform coefficients may also be grouped by splitting the transform coefficients 
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into nibbles; and grouping the nibbles into layers. Here, higher and lower nibbles 
may be grouped separately into layers. The transform coefficients may further be 
grouped by splitting the transform coefficients into crumbs; and grouping the nibbles 
and crumbs into layers. Additionally, the embodiments may further comprise entropy 
coding a second number of layers to generate a second description of compressed 
data. The embodiments may further comprise entropy coding each additional number 
of layers to generate a master inventory of compressed data. The embodiments may 
further comprise arranging the quantized transform coefficients prior to generating the 
energy distribution. 

[0010] In a further embodiment, a method for generating compressed data based on 

quantized transform coefficients of the data comprises accessing an inventory of 
multiple layers of compressed data generated based on an energy distribution of the 
quantized transform coefficients; and extracting a selected number of layers from the 
inventory to generate the compressed data. 
[0011] In still a further embodiment, an apparatus for generating compressed data 

based on quantized transform coefficients of the data comprises means for accessing 
an inventory of multiple layers of compressed data generated based on an energy 
distribution of the quantized transform coefficients; and means for extracting a 
selected number of layers from the inventory to generate the compressed data. 
[0012] In yet another embodiment, an apparatus for generating compressed data 

based on quantized transform coefficients of the data comprises a storage medium 
configured to store an inventory of multiple layers of compressed data generated 
based on an energy distribution of the quantized transform coefficients; and a 
selection module coupled to the storage medium and configured to extract a selected 
number of layers from the inventory to generate the compressed data. 

BRIEF DESCRIPTION OF THE DRAWINGS 
Various embodiments will be described in detail with reference to the 
following drawings in which like reference numerals refer to like elements, wherein: 

Figures 1 A and IB show adaptively sized blocks and sub-blocks for 
ABSDCT; 

Figure 2 shows example target applications; 



[0013] 
[0014] 
[0015] 
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[0016] Figure 3 shows an example of a multiple description compression system for 

generating and playback of images; 
[0017] Figures 4A and 4B show an example encoder and server for a multiple 

description compression system; 
[0018] Figures 5A and 5B show methods for a generating one or more descriptions of 

compressed data; 

[0019] Figure 6 shows an example energy distribution in tiers of AC coefficients in a 

16x16 block; 

[0020] Figures 7A to 7D show various masks for generating layers; and 

[0021] Figure 8 shows an example layout of bit stream for a transform based 

compression system. 

DETAILED DESCRIPTION 
[0022] Generally, embodiments described below allow transform based compression 

system to generate multiple descriptions of compressed data from input video data 
stream. In the following description, specific details are given to provide a thorough 
understanding of the embodiments. However, it will be understood by one of 
ordinary skill in the art that the embodiments may be practiced without these specific 
detail. For example, circuits may be shown in block diagrams in order not to obscure 
the embodiments in unnecessary detail. In other instances, well-known circuits, 
structures and techniques may be shown in detail in order not to obscure the 
embodiments. 

[0023] Also, it is noted that the embodiments may be described as a process which is 

depicted as a flowchart, a flow diagram, a structure diagram, or a block diagram. 
Although a flowchart may describe the operations as a sequential process, many of the 
operations can be performed in parallel or concurrently. In addition, the order of the 
operations may be re-arranged. A process is terminated when its operations are 
completed. A process may correspond to a method, a function, a procedure, a 
subroutine, a subprogram, etc. When a process corresponds to a function, its 
termination corresponds to a return of the function to the calling function or the main 
function. 

[0024] Moreover, as disclosed herein, the term "video" refers to the visual portion of 

multimedia and will be used interchangeably with the term "image." A storage 
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medium may represent one or more devices for storing data, including read only 
memory (ROM), random access memory (RAM), magnetic disk storage mediums, 
optical storage mediums, flash memory devices and/or other machine readable 
mediums for storing information. The term "machine readable medium" includes, but 
is not limited to portable or fixed storage devices, optical storage devices, wireless 
channels and various other mediums capable of storing, containing or carrying 
instruction(s) and/or data. 

[0025] In addition, an input video or image data stream is typically composed of 

image frames. An image frame can generally be divided into slices, a slice can be 
divided into data blocks, and a data block can be divided into pixels which are the 
smallest units of an image. Each image frame comprises an integer number of slices 
and each image slice represents the image information for a set of 16 consecutive scan 
lines. In such case, each data block corresponds to a 16x16 pixel block across the 
image of the frame. Also, a frame may be separated into even and odd slices, thereby 
forming even half frame and odd half frame. In one embodiment, half frames are the 
fundamental packets of compressed data information that are processed by a decoder. 
Moreover, an image pixel can be commonly represented in the Red, Green and Blue 
(RGB) color component system. However, because the human eye is more sensitive 
to changes in luminance and less sensitive to changes in chrominance, the YCbCr 
color space is typically used in video compression to represent image pixels. The 
YCbCr color space is a linear transformation of the RGB components, where Y is the 
luminance component, and Cb and Cr are the color components. If a frame is 
separated into even/odd frames, there would be three even half frames and three odd 
half frames corresponding to the components Y, Cb and Cr. 

[0026] In the description above, a slice can represent a set of consecutive scan lines 

other than 16 consecutive scan lines. Also, a different color space with the same or 
different number of color components may be used to represent an image pixel in 
accordance with the invention. 

[0027] Furthermore, compression techniques typically are based on discrete cosine 

transform (DCT) in which the size of each data block is fixed. One dynamic image 
compression technique capable of offering significant compression while preserving 
the quality of image signals utilizes adaptively sized blocks and sub-blocks of 
encoded DCT coefficient data. This technique will hereinafter be referred to as the 
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adaptive block size discrete cosine transform (ABSDCT). The adaptive block sizes 
are chosen to exploit redundancy that exists for information within a frame of image 
data. The technique is disclosed in U.S. Pat. No. 5,021,891, entitled "Adaptive Block 
Size Image Compression Method And System." DCT techniques are also disclosed 
in U.S. Pat. No. 5,107,345, entitled "Adaptive Block Size Image Compression 
Method And System," and the use of the ABSDCT technique in combination with a 
Discrete Quadtree Transform technique is discussed in U.S. Pat No. 5,452,104, 
entitled "Adaptive Block Size Image Compression Method And System." The 
systems disclosed in these patents utilize intraframe encoding, wherein each frame of 
an image sequence is encoded without regard to the content of any other frame. 

[0028] Generally, each of the luminance and chrominance components is passed to a 

block interleaver (not shown). A 16x16 block is presented to the block interleaver, 
which orders the image samples within the 16x16 blocks to produce blocks and 
composite sub-blocks of data for DCT analysis. Figure 1 A shows an example, where 
one 16x16 DCT is applied to a first ordering, four 8x8 DCTs are applied to a second 
ordering, 16 4x4 DCTs are applied to a third ordering, and 64 2x2 DCTs are applied 
to a fourth ordering. The DCT operation reduces the spatial redundancy inherent in 
the image source. After the DCT is performed, most of the image signal energy tends 
to be concentrated in a few DCT coefficients. 

[0029] For the 16x16 block and each sub-block, the transformed coefficients are 

analyzed to determine the number of bits required to encode the block or sub-block. 
Then, the block or the combination of sub-blocks which requires the least number of 
bits to encode is chosen to represent the image segment. Figure IB shows an example 
where two 8x8 sub-blocks, six 4x4 sub-blocks, and eight 2x2 sub-blocks are chosen to 
represent the image segment. The chosen block or combination of sub-blocks is then 
properly arranged in order. The DCT coefficient values may then undergo further 
processing such as, but not limited to quantization and variable length coding. 

[0030] For purposes of explanation, the transform based compression system for 

generating multiple descriptions or layers of compressed data will be discussed with 
reference to ABSDCT algorithm. However, it would be apparent to those skilled in 
the art that the invention is not limited to the use of ABSDCT. Other mathematical 
transforms such as, for example, DCT, Hadamard transform and Integer transform 
may also be used. 
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[0031] Generally, ABSDCT based compression supports encoding of up to 10-bit 

4:4:4 1920x1080 images. However, the ABSDCT algorithm is inherently scalable 
and can handle higher bit-depths with expanded bit-widths. It can also compress 
images of any size, including 4Kx4K, since it is a block-based compression. Given 
this flexibility of ABSDCT, a system compresses, for example, a 4Kx2K, 12-bit, 
4:4:4 image for visually lossless image quality. The resulting bit stream of block-wise 
DCT coefficients would, if completely decoded, produce the DC playback sequence. 
This bit stream is grouped and arranged such that lower resolution sequences can be 
extracted using simple operations of cropping. Such system results in a multiple 
description compression system. 

[0032] More particularly, the input data may be encoded once to generate a complete 

bit stream or a "Master Inventory." Within the master inventory are multiple levels of 
inventories. Here, a high level inventory may comprise one ore more lower level 
inventories. Typically, the highest level inventory would be the master inventory. 
Also, each inventory comprises a compressed bit stream that may caters to a different 
target application. 

[0033] Figure 2 shows some target applications such as Digital Cinema, High 

Definition Television (HDTV), Standard Television (SDTV), Digital Satellite System 
(DSS) and thumbnails that operate at compressed bit streams of different resolutions 
and bit rates. Other applications include, but are not limited to, Digital Versatile Disc 
or Video Disc (DVD), Advanced Television Systems Committee (ATSC), Digital 
Video Broadcast (DVB). As shown, a source data may have a format of 10 bit, 4:4:4 
and 1920x1080x24 or higher resolution. Digital Cinema requires a resolution of 
1920x1080x24, a frame rate greater or equal to 10 bit 4:4:4 and bit rate of 30-200 
Mbps. HDTV requires a resolution of 1920x1080x24, a frame rate of 8 bit 4:2:0 and 
bit rate of 15-19 Mbps. SDTV requires a resolution of 720x486x24, a frame rate of 8 
bit 4:2:0 and bit rate of 1.8-15 Mbps. DSS requires a resolution of 352x240x24, a 
frame rate of 8 bit 4:2:0 and bit rate of 3-7 Mbps. Thumbnails require a resolution of 
1 12x64x24, a frame rate of 8 bit 4:2:0 and bit rate of 200 Mbps. 

[0034] Figure 3 shows an example system 300 for generation and playback of image 

sequences based on a multiple description compression system. Generally, a hub 310 
generates a master inventory or a portion of the master inventory comprising of 
multiple inventories. Hub 310 may then output the master inventory or the portion to 
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a distribution center 320. Distribution center 320 may then output various 
inventories, each catering to a different target application or presentation system 330 
for playback. As shown, there may be one or more presentation systems 330, catering 
to the same or different target applications. Here, hub 310 and distribution center 320 
may be implemented together. Alternatively, hub 310 and distribution center 320 
may be implemented as separate structures or at separate locations. Similarly, 
distribution center 320 and presentation system 330 may be implemented together. 
Also similarly, distribution center 320 and presentation system 330 may be 
implemented as separate structures or at separate locations. If hub 310 and 
distribution center 320 or if distribution center 320 and presentation system 330 are 
implemented at separate location, data may be transmitted using a wireless medium, a 
non- wireless medium, a portable storage medium or a combination thereof. 

[0035] More particularly, Hub 310 may include an encoder 400 shown in Figure 4A 

that receives digital video information, such as a motion image sequence, to be 
compressed. Encoder 400 is capable of compressing input data into multiple 
descriptions of compressed data and comprises a transform module 410, a 
quantization module 420, a layering module 430 and an entropy coder 440. In one 
embodiment, transform module 430 may be a DCT module using a DCT based 
algorithm and more specifically, may be a DCT module using ABSDCT. However, 
other transform based algorithms may also be applicable. 

[0036] Typically, transform module 410 converts the input data information from 

spatial to frequency domain and generates transform coefficients. Quantization 
module 420 quantizes the transform coefficients. Layering module 430 generates an 
energy distribution of the quantized transform coefficients and groups the transform 
coefficients into layers based on the energy distribution. Entropy coder 440 entropy 
codes a number of layers to generate an inventory of a certain level. Entropy coder 
440 may encode different numbers of layers to generate inventories of different 
levels, wherein each inventory comprises different layers of compressed data. 
Entropy coder 440 may further entropy code additional numbers of layers to generate 
each possible inventory, thereby generating a master inventory. Here, various 
variable length encoder may be used as entropy coders, such as for example, Golomb 
coder, Rice Coder, Huffman engine or a combination thereof. 
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[0037] Distribution center 320 may comprise a server 450 shown in Figure 4B that 

provides compressed data to presentation system 330. Server 450 may comprise a 
storage medium 460 and a selection module 470. Storage medium 460 stores an 
inventory of compressed data received from hub 310. The inventory may be the 
master inventory of may be a lower level inventory. Selection module 470 then 
extracts a selected number of layers from the inventory to generate a description of 
compressed data. For example, the selection may be based on the target application. 
The generated description of compressed data may then be used for playback at 
presentation system 330. It is to be noted that the generated descriptions of 
compressed data may be sub-sampled to provide a format for playback at presentation 
system 330. 

[0038] Presentation system 330 comprises a decoder that decompresses the received 

image using a decompression algorithm that is inverse to the compression algorithm 
used at encoder 400. For example, if image compression is based on the ABSDCT, 
image is variable length decoded, inverse quantized and inverse DCT processed to 
enable display of the digital image. 

[0039] The operations will be described more in detail below in Figures 5 A and 5B. 

However, it should first be noted that a more typical hub 310 may comprise other 
elements such as a storage medium (not shown) to store one or more inventories 
generated by encoder 400. Hub 310 may also comprise a selection module to extract 
a selected number of layers to be output to distribution center 320. In addition, a 
processor (not shown) may be implemented to control one or more elements of 
encoder 400. Such processor may be implemented as part of encoder 400 or may be 
implemented outside of encoder 400. Similarly, a processor (not shown) may also be 
implemented to control one or more elements of distribution center 320. Such 
processor may be implemented as part of server 450 or may be implemented outside 
of server 450. Accordingly, a more typical distribution center 320 may comprise 
other elements as well. 

[0040] More specifically, Figure 5A shows an example method 500 for generating 

multiple descriptions of compressed data. In method 500, transform coefficients are 
generated (520) from input data and quantized (520). Here, the transform coefficients 
may be generated by transform module 410 using for example DCT or ABSDCT 
algorithm and may be quantized by quantization module 420 based on block size 
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assignment. In order to extract lower resolution sequences from a compressed bit 
stream, the transform coefficients may be grouped and arranged in the order of 
significance. This order may be determined based on the resolution and bit rate 
requirements of target applications. The order may be optimized for image quality 
using traditional and ABS-specific, if applicable, rate-distortion techniques. 
[0041] Referring back to Figure 5, an energy distribution of the quantized transform 

coefficients is generated (530). Figure 6 shows an example of an energy distribution 
in tiers of AC coefficients in a 16x16 block when the transform coefficients are 
generated using, for example, ABSDCT based algorithm. The adaptive block size 
assignment algorithm is, by itself, an indicator of the rate-distortion characteristics of 
an image. Therefore, statistics may be compiled (histogram and entropy calculated) 
for the AC coefficients on each tier. Figure 7A-7D show examples of layers defined 
based on the statistics for a 2x2 sub-block, a 4x4 sub-block, a 8x8 sub-block and a 
16x16 block. Based on the energy distribution, the transform coefficients are grouped 
(540) into layers. 

[0042] In one embodiment, the AC coefficients may first be arranged in zig-zag order 

and their energy distribution across tiers may be used to group them into layers. The 
coefficients may further be split into multi-bit units and grouped into layers. The 
coefficients may be split into nibbles and grouped into layers. Here, higher and lower 
nibbles may be grouped separately into layers. This makes it more efficient to extract 
lower precision streams. The coefficients may also be split into nibbles and crumbs, 
and grouped into layers. 

[0043] The energy distribution may be generated by layering module 430 or by a 

processor. The transform coefficients may also be grouped by layering module 430 or 
by a processor. A number of layers is entropy coded (550) to generate an inventory of 
compressed data. Here, entropy coder 440 may generate the description of 
compressed data. In addition, different number of layers may be entropy coded to 
generate different inventories. If each possible inventories is generated, a master 
inventory is generated. A specific inventory may then be generated by extracting 
lower precision stream. 

[0044] For example, Figure 5B shows a method 560 for generating a specific 

inventory of compressed data based on quantized transform coefficients of the data. 
In method 560, an inventory of compressed data that was generated based on an 
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energy distribution of the quantized transform coefficients is accessed (570). A 
selected number of layers is then extracted (580) from the inventory to generate one 
description of compressed data. To generate a different description of data, a 
different number of layers is extracted, thereby providing multiple descriptions of 
compressed data. Here, an appropriate layer or layers are chosen to meet the bit rate 
requirements of the target application. A storage medium such as storage medium 
460 may store the inventory and selection module 470 may extract the selected 
number of layers. 

[0045] Figure 8 shows an example of the bit stream layout that may be stored in a 

storage medium as an inventory. In the example, 10-bit transform coefficients are 
split into a higher nibble, lower nibble and 2 least significant bits (LSBs). DSS 
requires the higher and lower nibbles of layers 0 to 2 to generate 8-bit 4:2:0 
352x240x30 image. SDTV requires the higher and lower nibbles of layers 0 to 3 to 
generate 8-bit 4:2:0 720x486x30 image. HDTV requires the higher and lower nibbles 
of layers 0 to 6 to generate 8-bit 4:2:0 1920x1080x30 image. Digital Cinema requires 
the higher and lower nibbles of layers 0 to 7 as well as 2 LSBs or layers 0 to 7 to 
generate 10-bit 4:4:4 1920x1080x30 image. It is to be noted that other formats of 
lower and/or higher resolution may also be generated. 

[0046] For example, Figure 8 additionally includes a thumbnail inventory that may be 

generated by compressing the scaled (l/256 th ) image formed by the means of the 
blocks (obtained from BSA). This image is compressed with ABSDCT. The target 
bit rate for this layer is 200Kbps for a 112x64x24fps image sequence. The thumbnail 
inventory is similar to preview option available with some formats like Encapsulated 
Postscript. Note that formats other than DSS, SDTV, HDTV, Digital Cinema and 
Thumbnail, may also be added to the master inventory by using different 
combinations of layers, nibbles and LSBs. 

[0047] Accordingly, multiple inventories of layers of compressed data are generated 

to meet the requirements of target applications. Thereafter, necessary layers are 
extracted or cropped from the multiple layers to provide a specific description of 
compressed data for a target application. In an alternative embodiment, an archival 
compression may be performed to generate an archival inventory. To generate a 
specific inventory for a target application, the bit stream is decoded to an archival 
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quality playback sequence. The DC playback sequence may then be extracted from 
the bit stream. 

[0048] More particularly, the archival compression may be generated by transform 

based compression using a reference quantization step. Input image may be discrete 
cosine transformed, quantized using a reference quantization step and variable length 
encoded to generate the archival compressed bit stream. To extract a specific 
inventory, the compressed bit stream is variable length decoded and re-quantized. For 
example if the reference quantization step is A with corresponding value a and the 
target application requires a quantization step of B with corresponding value b, the 
quantization step for the re-quantization would be determined based on the required 
scaling of the reference quantization step. Here, it would be b/a. The bit stream may 
then be variable length coded and sent to a presentation system for playback. 

[0049] It should be apparent to those skilled in the art that the elements of encoder 

400 and/or a server 450 may be rearranged without affecting the operations. Also, 
embodiments may be implemented by hardware, software, firmware, middleware, 
microcode, or any combination thereof. When implemented in software, firmware, 
middleware or microcode, the program code or code segments to perform the 
necessary tasks may be stored in a machine readable medium such as storage medium 
460 respectively, or in a separate storage(s) not shown. A code segment may 
represent a procedure, a function, a subprogram, a program, a routine, a subroutine, a 
module, a software package, a class, or any combination of instructions, data 
structures, or program statements. A code segment may be coupled to another code 
segment or a hardware circuit by passing and/or receiving information, data, 
arguments, parameters, or memory contents. Information, arguments, parameters, 
data, etc. may be passed, forwarded, or transmitted via any suitable means including 
memory sharing, message passing, token passing, network transmission, etc. 

[0050] Therefore, the foregoing embodiments are merely examples and are not to be 

construed as limiting the invention. The description is intended to be illustrative, and 
not to limit the scope of the claims. As such, the present teachings can be readily 
applied to other types of apparatuses and many alternatives, modifications, and 
variations will be apparent to those skilled in the art. 

What is claimed is: 



